\section{Document Conventions}

Literals are denoted with \lit{monospace}.  Syntatic categories are denoted with \syncat{italics} and are all lowercase.  
Identifiers, integers, and floats are represented by \syncat{Id}, \syncat{Int}, and \syncat{Float} respectively.  
Optional items are indicated with ``opt'' in subscripts following the item, ex. \syncat{\opt{optional-item}}.  Sometimes the syntatic categories are enumerated in the suffix, ex. \syncat{item-1}, for ease of reference.  
Section numbers to the right of the productions indicate the location of syntatic categories not defined in the same subsection.

\section{Lexical Conventions}

\subsection{Comments}
Comments begin with the characters /* and continue until */.

\subsection{Identifiers}
Identifiers consist of letters, digits, and underscores.  The first character must be a letter.  Identifiers are case sensitive.

\subsection{Keywords}
The following identifiers are reserved keywords and may not be used for any other purpose.

\begin{tabular}{p{4in} l}
\lit{module}\\
\lit{size}&\InlineSingleTestNote{test-flip5a}\\
\lit{for}\\
\lit{t}\\
\lit{end}\\
\lit{kernel}\\
\lit{pi}&\InlineSingleTestNote{test-constant-pi}\\
\lit{e}&\InlineSingleTestNote{test-constant-e}\\
\lit{sin}&\InlineSingleTestNote{test-sin}\\
\lit{cos}&\InlineSingleTestNote{test-cos}\\
\lit{exp}&\InlineSingleTestNote{test-exp}\\
\lit{pragma}\\
\lit{input}
\end{tabular}

\lit{t} and \lit{end} are not used but are reserved for later\TestNote{\RefTest{fail-reserved-word-t} and \RefTest{fail-reserved-word-end}}.

\subsection{Constants}
There are two types of constants, int constants and float constants.

\subsubsection{Integer Constants}
An int consists of one or more digits.

\subsubsection{Float Constants}
A float consists of a decimal point and at least one digit.  The precision of the float is compiler dependent and may even be implemented as an integer using scaling.  An int can be implicitly casted as a float, but not vice versa.

\subsubsection{Built-in Constants}
$e$ and $pi$ are built-in constants which are approximately 2.71828 and 3.14159 respectively%
\TestNote{ \RefTest{test-constant-e} and \RefTest{test-constant-pi}}.  
The accuracy depends on the precision of float used by the compiler.

\subsection{Program Parameters}
The input and output sources of the program are specified by \$1, \$2, etc.  For command-line applications \$1 corresponds to the first parameter, \$2 the second, etc.  How they are used will determine whether they are input or output.  They may not be both \SingleTestNote{fail-in-out-param}.  Every input must be declared with its dimension.\\

\newsyn{input-decl}{
	\lit{input} Param dimensions \lit{;}& \textup{\S\ref{sec:dimensions}}\\
}

\section{Program}
A program consists of module definitions, kernel function definitions, activation function definitions, and synaptic connections.\\

\newsyn{program}{
	\NOTHING \\
	input-decl program & \textup{\S\ref{sec:input-decl}}\\
	module-def program & \textup{\S\ref{sec:module-def}}\\
	%module-decl program& \textup{\S\ref{sec:module-decl}}\\
	activation-def program& \textup{\S\ref{sec:activation-def}}\\
	kernel-def program& \textup{\S\ref{sec:kernel-def}}\\
	synap-connection program& \textup{\S\ref{sec:synap-connection}}
}

\subsection{Module Definition}  % ======================================================================================================


Neurons can only be defined in modules.  %Multiple instances of a module can be used with module declarations (section \ref{sec:module-decl}).
There are three exclusive types of neurons in a module: input neurons, output neurons, and inner neurons.  Input neurons receive external
signals, output neurons send external signals, and inner neurons are encapsulated in the module.\\

\newsyn{module-def}{
  \lit{module} Id neurons-1 \RDIRECT neurons-2 \LBRACE module-body \RBRACE
}
\syncat{Id} is the name of the module.
\syncat{neurons-1} and \syncat{neurons-2} are the list of input and output neurons respectively.\\

\newsyn{neurons}{
  Id \opt{dimensions} \\
  Id \opt{dimensions} \lit, neurons
}

The inner and output neurons are defined with an activation expression inside of the module using \syncat{neuron-def}.
The activation expression of an input neuron is defined by synaptic connections outside of the module using \syncat{synap-connection}.

%All modules are globally defined, however, instances of a module can be declared inside another.  Recursive instantions are not allowed.\\

\newsyn{module-body}{
  \NOTHING \\
  neuron-def module-body & \textup{Within a module}\\
  synap-connection module-body& \textup{Between modules, \S\ref{sec:synap-connection}}\\
  %module-decl module-body & \textup{\S\ref{sec:module-decl}}\\
}

\newsyn{neuron-def}{
	Id \opt{dimensions} \LDIRECT expression\SEMI\\
	Id \opt{dimensions} \LDIRECT expression \lit{for} for-list \SEMI  & \textup{\S\ref{sec:for-list}}\\
}

The variable iterators can only be used in \syncat{expression}.  
\syncat{dimensions} is used to specify the size of the array of neurons and must be equal in size of the expression that is being iterated over.

Modules may be used directly:

\begin{verbatim}
modulename1.input << modulename2.output;
\end{verbatim}

or may be instantiated:

\begin{verbatim}
modulename1 mods[2];
mods[1].input_neuron << mods[2].output_neuron;
\end{verbatim}



%
%\subsection{Module Declaration}  % ======================================================================================================
%
%Once a module has been defined, multiple instances can then be declared.\\
%
%\newsyn{module-decl}{
% Id-1 Id-2 \opt{dimensions} \SEMI
%}
%
%where \syncat{Id-2} is a new instance of the module \syncat{Id-1}.
%
%\begin{verbatim}
%module module1 x >> y
%{
%	y = act( .5 * x );
%}
%\end{verbatim}
%
%\begin{verbatim}
%module module2 input[100] >> output
%{
%	module1 mod[100];
%	mod[i].x << input[i] for i = 1:end;
%	output << act( mod[i].y, gauss );
%}
%\end{verbatim}
%
%
%Modules may also contain instantiations of other modules.

\subsection{Function Definitions}   % ======================================================================================================

There are two types of functions allowed in Synapse: activation functions and kernel functions.  
Activation functions take and returns a scalar while kernel functions generate matrices that fit the context referenced.

\subsubsection{Activation Function Definitions}

Activation functions take a single scalar and returns a single scalar\SingleTestNote{fail-afun-mat}.

\newsyn{activation-def}
{
  Id-1 \LPAREN Id-2 \opt{fparams} \RPAREN \EQUALS expression \SEMI }
  
\syncat{Id-1} is the name of the function and \syncat{Id-2} is the name of the local input scalar.  \syncat{fparams} are optional\SingleTestNote{test-afun1}
and may be used to define parameters of type float with their default values\TestNote{\RefTest{test-afun2} and \RefTest{test-afun3}}.

\newsyn{fparams}
{
  /* nothing */ \\
  \SEMI fparam-list
}

\newsyn{fparam-list}{
	Id \EQUALS Float \\
	Id \EQUALS Float \COMMA fparam-list\\
}

\subsubsection{Kernel Function Definition}  % ======================================================================================================

Kernel definitions\SingleTestNote{test-kernel1d} may only be used directly to the right of a convolution operation (\S\ref{sec:convolution}).

\newsyn{kernel-def}{
	\lit{kernel} Id \LPAREN id-list \opt{fparams} \RPAREN \EQUALS expression \SEMI &\textup{\S\ref{sec:fparams},\ref{sec:expression}}
	}
	
\syncat{id-list} contains the comma-delimited names for the indices that may be referenced in \syncat{expression}.  
The first index refers to the first dimension (the row if 2D), the second index refers to the second dimension, etc.  If $w$ is the number of cells in a dimension, then the indices are enumerated from $-\frac{w-1}{2}$ to $\frac{w-1}{2}$ while incrementing by 1.  
Therefore, if the dimension is even, then the index values will not be an integer.

The Gabor filter can be implemented as:

\lstset{frame=single,numbers=left,numbersep=4pt,language=Synapse,keywordstyle=\color{blue},commentstyle=\color{DarkGreen}}
\begin{lstlisting}
kernel gabor(x,y;lambda=1,theta=0,psi=0,sigma=1,gamma=0) = 
   exp(-((x*cos(theta) + y*sin(theta))^2+gamma ^ 2 * (-x*sin(theta)
         + y * cos(theta))^2)/(2 * sigma^2)) 
 * cos( 2*pi*(x*cos(theta) + y*sin(theta))/lambda+psi);
\end{lstlisting}


\subsection{Inter-Module Synaptic Connections} % ===========================================================================

The synaptic connections are used to connect the input and output neurons between modules.

\newsyn{synap-connection}{
  neuron-scoped \opt{dimensions} \LDIRECT expression \SEMI &\textup{\S\ref{sec:dimensions}}\\
  neuron-scoped \opt{dimensions} \LDIRECT expression \lit{for} for-list \SEMI & \textup{\S\ref{sec:for-list}}
  \lit{Param} \opt{dimensions} \LDIRECT expression \SEMI\\
  \lit{Param} \opt{dimensions} \LDIRECT expression \lit{for} for-list
}

\newsyn{dimensions}{
 \LBRACK const-int-list \RBRACK
}

\newsyn{const-int-list}{
  const-int-expr & \textup{ \S\ref{sec:primary-expression} }\\
  const-int-expr \lit{,} const-int-list
}

See \S\ref{sec:neuron-scoped} for the definition of \syncat{neuron-scoped} and for information on scoping.  The last two definitions, with \lit{Param}, can only be used if the connection is made in the global scope.

\section{Expressions} % ======================================================================================================

The subsections below appear from highest to lowest precedence.  Operators within a subsection have equal precedence.

\newsyn{expression}{
	primary-expression \\
  \LPAREN expr \RPAREN            \\
  expr \PLUS   expr              \\
  expr \MINUS  expr              \\
  expr \TIMES  expr              \\
  expr \DIV    expr              \\
  expr \POW    expr              \\
  \MINUS expr						        \\
  expr \CONV kernel-call	& \textup{\S\ref{sec:kernel-call}} \\
  \lit{pi}                   \\
  \lit{e}                     \\
  \lit{exp} \LPAREN expr \RPAREN \\
  \lit{sin} \LPAREN expr \RPAREN \\
  \lit{cos} \LPAREN expr \RPAREN \\
}


\subsection{Primary expressions}

Primary expressions include the below syntatic category plus kernel function calls.  
Kernel function calls can only appear to the right of a convolution operator (\ref{sec:convolution}).\\

\newsyn{primary-expression}{
	Float\\	
	Int\\	
	indexable-expression \opt{indices} \\
	activation-call & \textup{\S\ref{sec:activation-call}}
}

\newsyn{indexable-expression}{
	Param \\	
	Id \\ 
	scoped-neuron\\
}

\newsyn{indices}{
	\lit[ index-list \lit]
}

\newsyn{index-list}{
	index-num \\
	index-num \lit, index-list
}

One-based indexing is used. %A single colon (ex. [:]) references all of the elements in the respective dimension.  
\syncat{index-num-1} is the first number in the range when expanded and
\syncat{index-num-2} is the last.  If specified, the middle number, \syncat{const-int-expr}, specifies the increment, otherwise
each consecutive number is included in the range.  \\

\newsyn{index-num}{
	const-int-expr \\
	%\lit{begin}\\
	%\lit{end}
}

In the future, \lit{end} will be added to index-num and indices will be able to include spans.

\newsyn{const-int-expr}{
	Int \\
	Id\\
	\lit(  const-int-expr \lit) \\
	const-int-expr \lit+ const-int-expr \\
	const-int-expr \lit- const-int-expr \\
	const-int-expr \lit* const-int-expr \\
	\lit- const-int-expr & \textup{\InlineSingleTestNote{test-flip5c}}\\
	size-macro & \textup{ \S \ref{sec:size-macro} \InlineSingleTestNote{test-flip5b}}
}

Index expressions are a subset of regular expressions which enforces that indices are only integers\TestNote{\RefTest{test-flip5a} and \RefTest{test-flip5b}}.
\syncat{Id} in this case must be an index defined by a for macro.\\

The operator . and subscripting group left to right.


\subsection{Convolution operator}
\subsubsection{ \syncat{expression} \lit{**} \syncat{kernel-call} }
\label{sec:convolution}

The binary operator \lit{**} indicates convolution.  The expression to the left must evaluate to a matrix of fixed size.  On the right, a kernel function is referenced and a matrix is generated that matches the dimension of the expression on the left.  
A convolution performs a pointwise multiplication on the matrices and 
sums the elements of the resulting matrix \SingleTestNote{test-kernel1d}.

\subsection{Unary operator}
\subsubsection{ \lit- \syncat{expression}}
The unary operator \lit- negates the expression and has the same type.  If the expression is a matrix, then every element is negated \SingleTestNote{test-matrixnegate}.

\subsection{Exponential operator}

\subsubsection{\syncat{expression-1} \lit{ \textasciicircum } \syncat{expression-2}}
The binary operator \lit{ \textasciicircum }  indicates expression-1 being raised to the power of expression-2 \SingleTestNote{test-scalarpowdiff}.  
expression-1 must be a float, an int, or a matrix.  If it is a matrix, then each element in expression-1 is raised to the power of expression-2.
expression-2 must be a float or an int.  The result is either a float or a matrix of float.

\subsection{Multiplicative operators}
\subsubsection{\syncat{expression-1} \lit* \syncat{expression-2}}
The binary operator \lit* indicates pointwise multiplication.  If both operands are matrices, then the element-by-element product is returned.  In this case, both operands must have the equal dimensions.  Otherwise, at least one of the expressions is a scalar.  If either of the operands is a matrix, then the result is a matrix; else if either of the operands is a float, then the result is a float; otherwise both of the operands is an int and an int is returned.

\subsubsection{\syncat{expression-1} \lit/ \syncat{expression-2}}
The binary operator / indicates pointwise division.  The same size considerations apply as for multiplication.  If either operand is a matrix the result is a matrix \TestNote{\RefTest{test-matrixmatrix-div}, \RefTest{test-matrixfloat-div1} and \RefTest{test-matrixfloat-div2}}; otherwise the result is a float.  Integer division does not exist in Synapse\SingleTestNote{fail-int-div}.  An expression that contains division must not be used when constant integers are required, as when defining the size of a matrix.

The results for division by zero are currently undefined, however, it may be defined in future versions of Synapse.

\subsection{Additive operators}
\subsubsection{\syncat{expression-1} \lit+ \syncat{expression-2}}
The binary operator \lit* indicates pointwise addition\TestNote{\RefTest{test-matrixadd}, \RefTest{test-matrixadd2} and \RefTest{test-matrixadd3}}.  The same size and type considerations apply as for multiplication.

\subsubsection{\syncat{expression-1} \lit- \syncat{expression-2}}
The binary operator \lit- indicates pointwise subtraction.  The same size and type considerations apply as for multiplication.

\section{Function Calls} % ======================================================================================================
\subsection{Built-in functions}
\subsubsection{sin}
\lit{sin} \LPAREN x \RPAREN calculates the sine of $x$ in radians.

\subsubsection{cos}
\lit{cos} \LPAREN x \RPAREN calculates the cosine of $x$ in radians.

\subsubsection{exp}
\lit{exp} \LPAREN x \RPAREN calculates $e^x$.

\subsection{User-defined functions}

Activation functions may be called anywhere \ref{sec:expression} can be used.

\newsyn{activation-call}{
	Id \LPAREN \RPAREN \\
	Id \LPAREN fparam-list \RPAREN & \textup{ \S\ref{sec:fparam-list} } \\
}

Kernel functions may be referenced directly after a convolution operator.

\newsyn{kernel-call}{
	Id \LPAREN \RPAREN \\
	Id \LPAREN fparam-list \RPAREN & \textup{ \S\ref{sec:fparam-list}} \\
}

The usefulness of kernel functions will be limited until spans are allowed in index expressions.

\section{Macros} % ======================================================================================================

\subsection{\lit{for} macro}

The for-macro makes it easier to connect a large number of modules, matrices of modules, and matrices of neurons.\\

\newsyn{for-list} {
  for-expression \\
  for-expression for-list
  }

\newsyn{for-expression}{
  Id \EQUALS \LBRACK index-expression \RBRACK\\
}


\newsyn{span}{
	%index-num\\
	%\lit:\\
	index-num-1 \lit: index-num-2 & \textup{ \S \ref{sec:index-num} }\\
	index-num-1 \lit: const-int-expr \lit: index-num-2
}

Synaptic connections that use the for-macro will be evaluated for every combination of values in the Ids ranges%
\TestNote{\RefTest{test-copy5a},\RefTest{test-copymat} and \RefTest{test-copymat2}}.  The for-variable must
appear in both the source and destination of the synaptic connection \SingleTestNote{fail-for-1}.

Used as an index to a module or neuron, it stands the smallest (ie. 1) and largest
number that are well defined in that module or neuron respectively.

\subsection{\lit{size} macro}
The size macro returns the size of a module or macro in the specified dimension.\\

\newsyn{size-macro}{
	\lit{size} \lit( indexable-expression \lit, Int \lit) & \textup{\S\ref{sec:indexable-expression}}
}

This macro can currently only be used when specifying indices\SingleTestNote{test-flip5b} and cannot be used in spans (including in for-loops).
	
\section{Scope} % ======================================================================================================

The scope of neurons are local to the current module.  The neurons may be specified in any order.  Neurons within or
between modules may have circular or recurrent connections.

When connecting input and output neurons between modules, the module for which the neuron belongs must be specified.

\newsyn{neuron-scoped}{
	Id-1 %\opt{indices} 
	\DOT Id-2 & \textup{ \S \ref{sec:indices}}
}

\syncat{Id-1} specifies the module %or matrix of modules 
and \syncat{Id-2} specifies the neuron contained in the module \syncat{Id-1}.  
%\syncat{indices} is used if and only if \syncat{Id-1} is a matrix of modules.
While modules may be nested, only local neurons may be input or output neurons.  
Hence, only a single module is ever needed to reference a neuron.

All functions and module definitions have global scope.  An activation function may only reference functions defined before it%
\TestNote{\RefTest{test-afun-chain} and \RefTest{fail-afun-order}}.  Activation functions may not be defined recursively.
Neither function definitions may contain references to neurons.
%Module definitions may only contain nested instances of modules defined before it.
%Neither functions nor modules may be recursive.

Synaptic connections can connect modules and neurons regardless of location.

\section{Concurrency} % ======================================================================================================

Unlike traditional programming languages, all of the values at the synaptic connections are calculated concurrently.
Each synapse connection takes a single time step and at time $t$ only the values from time $t-1$ are used for the
calculations.

The order of execution at each time step is\TestNote{essentially every test case, but in particular \RefTest{test-temporaloffset}}:
\begin{enumerate}
\item Input parameters are updated.
\item Neurons are updated using their corresponding synaptic connection.
\item Output parameter values are written.
\end{enumerate}



Since it takes a while for the values from the input to be propagated throughout the program, the initial values are to
be defined by the compiler runtime options.  The compiler or interpreter must support the option of initialization of the neurons to zero.
Other options, such as initialization by use of random distributions may also be supported.
When the program starts writing output is also compiler or interpreter defined.  It must at minimum support the option to
start writing output as soon as it starts running, which means the initial output will be garbage.

While various inputs are supported, sequences of images or videos are well suited for reading and writing a large number of values.

\section{Future Additions} % ======================================================================================================

The following additions are planned for Synapse.

The support for the size macro will be expanded.  The keyword 'end' can be used in spans.

At any time $t+1$, the current version of Synapse only allows values from time $t$ to be referenced.  
Future versions will allow any $t$ or older neuron values to be referenced.  This will be support by making $t$ a keyword that can be used in arrays.  For example, $x$ would be the same thing as writing $x[t]$, $y[t,1:10]$ would be the same thing as $y[1:10]$, and $x[t-1]$ and $y[t-2,1:10]$ would refer to previous versions.

Some form of inline switch statements will be allowed in functions.

A dimension macro will be added that could be referenced in weight definitions.  A way of automatically normalizing the dynamic kernels will be added.  For example, Z may become a macro that stands for the sum of the weights of the current kernel.

Matrix constants can be defined in the form of: [[ 1, 2, 3, 4; 5, 6, 7, 8 ]].

Support for spike trains will be added by adding support for booleans and by adding support for Poisson spike generators.

%\section{Examples} % ======================================================================================================
%\subsection{Image Sharpening} 
%The following program creates a network that models a network of on-center surround cells.  It sharpens the video that is passed.
%
%\begin{verbatim}
%kernel sharpen( x,y; pos_sigma=1.0, neg_sigma=2.0 ) = 
%	1 / 2 * pi * pos_sigma ^ 2 
%	* exp( -.5 * ( x^2/pos_sigma^2 + y^2/pos_sigma^2 - 2*x*y / pos_sigma^2 ) )
%- 1 / 2 * pi * neg_sigma ^ 2 
%* exp( -.5 * ( x^2/neg_sigma^2 + y^2/neg_sigma^2 - 2*x*y / neg_sigma^2 ) );
%
%module OnCenterSurround in[7,7] >> out
%{
%	out << in ** sharpen();
%}
%
%OnCenterSurround on[ (size($1,1)-6)/2, (size($1,2)-6)/2 ];
%
%on[x-3,y-3].ker[:,:] << $1[x:x+6,y:x+6] for x=1:2:end, y=1:2:end;
%$2[x,y] << on[x,y].out for x=begin:end, y=begin:end;
%\end{verbatim}
%
%\subsection{Image Mirror} 
%
%The following one-liner mirrors the video.
%
%\begin{verbatim}
%$2[ y,size($1,2)-x+1] << $1[y,x] for x=begin:end, y=begin:end;
%\end{verbatim}
%
%Note that size(\$1,2) would not be able to be called inside of the index of \$2.
